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ABSTRACT 

This  paper  describes  a  method  for  detecting  and  segmenting  spherical  features  using  the 
gradient  angle  transform.  An  analysis  of  the  gradient  angle  for  ideal  spheres  is  presented,  with 
a  discussion  of  how  this  may  be  used  to  locate  the  boundaries  of  the  sphere.  The  algorithms 
used  by  a  program  which  detects  and  segments  spherical  features  are  then  presented.  The 
results  of  applying  the  program  to  images  with  naturally-occurring  spherical  features  are  given. 
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1  INTRODUCTION 


('nr  of  l hr  goals  of  computer  vision  rrsrarch  is  the  development  of  software  systems  for  the 
analvsis  of  aerial  pilot ographs  One  aspect  of  this  analysis  is  the  detection  and  segmentation  of 
various  features,  where  detection  refers  to  identifying  the  presence  of  a  feature  and 
segmentation  refers  to  locating  the  boundaries  of  the  feature.  The  information  produced  by 
these  programs  may  be  used  both  to  develop  a  high-level  interpretation  of  the  image  and  as 
entries  in  a  feature  database. 

Previous  researchers  have  developed  detector/segmenters  for  such  features  as  roads,  rivers,  and 
buildings.  These  detectors  have  largely  relied  on  image  edges  and  image  grey  levels  both  to 
detect  and  to  segment  the  features  [  1 , 2,5,10). 

We  have  developed  a  detector/segmenter  for  spherical  features  which  uses  the  direction  of  the 
image  gradient  for  segmentation.  This  value,  here  called  the  gradient  angle,  indicates  the 
direction  of  greatest  change  in  image  intensity.  Approximations  to  the  gradient  angle  have 
previously  been  used  in  certain  special  transforms  (for  example,  the  Hough  transform  for 
locating  circles).  The  detector/segmenter  presented  here  uses  continuity  of  the  gradient  angle  in 
segmentation,  an  approach  which  is  to  the  best  of  our  knowledge  original. 

We  feel  that  the  detector/segmenter  will  form  a  useful  part  of  a  photo  analysis  system.  We  also 
believe  that  the  angle-based  segmentation  methods  will  prove  important  for  other  types  of 
features. 

Section  2  of  this  report  describes  the  gradient  angle  and  explains  how  it  can  be  used  for 
segmentation.  The  section  concludes  with  an  analysis  of  the  gradient  angle  for  ideal  spheres 
under  point-source  illumination.  Section  3  outlines  the  algorithms  used  in  the  sphere 
detector/segmenter  program.  Section  4  gives  the  results  of  applying  the  program  to  both 
synthetically-generated  and  actual  spherical  surfaces.  Our  conclusions  are  presented  in  section 
5. 


2.  GRADIENT 
2.1.  Definitions 

The  gradient  of  a  point  in  an  image  is  a  measure  of  the  change  in  the  image  intensity  at  that 
point.  The  most  common  use  of  the  gradient  is  as  an  edge  detector;  points  of  high  gradient 
mark  edges  between  two  regions.  Many  edge-detecting  gradient  measures  have  been  described, 
including  the  Sobel,  Kirsch,  Prager,  and  Marr-Hildreth  detectors;  most  of  these  can  be 
calculated  using  a  convolution  (1,3,6,7,9,11). 

Define  the  quantities  GX  and  GY  as  the  gradient  in  the  X  and  Y  directions  for  a  particular 
point,  computable  by  any  of  several  convolutions;  for  example,  the  convolution  masks 
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produce  satisfactory  results.  Lsing  these,  the  gradient  magnitude  GM  and  gradient  angle  C,  \  arc 
computed  by  the  formulas; 


GM  “  sqrt(GX‘  ■+•  GY*) 


GA  =  lan  ‘(GY/GX) 

The  results  presented  in  this  paper  use  the  above  operations;  however,  these  are 
computationally  expensive.  We  have  found  that  approximating  GM  as  the  sum  of  the  absolute 
values  of  GX  and  GY,  and  GA  by  a  table  look-up  is  usually  satisfactory. 

2.2.  Use  in  Segmentation 

The  gradient  angle  has  proved  to  be  useful  for  certain  types  of  segmentation,  including  uses 
reported  in  [4].  The  value  of  the  gradient  angle  is  the  direction  of  greatest  change  in  pixel 
intensity.  Thus,  a  region  of  smoothly  increasing  intensity,  such  as  is  often  found  in  curved 
objects,  has  approximately  constant  gradient  angle.  Conversely,  a  region  of  approximately 
constant  intensity  has  irregular,  highly  textured  gradient  angle  due  to  slight  variations  in 
intensity.  This  makes  the  gradient  angle  of  an  image  an  excellent  space  to  search  for  curved 
features,  since  the  desired  features  form  large  regions  of  slowly-varying  value  while  planar 
features  are  broken  up.  Such  curved  features  are  among  the  more  difficult  to  detect  using 
traditional  segmentation  algorithms. 

2.3.  Special  Analysis  of  Sphere 

Consider  the  gradient  angle  transform  of  a  spherical  surface  of  radius  R  illuminated  by  a 
distant  point-source.  Using  a  simple  reflectance  model  as  in  [l],  the  intensity  of  a  point  on  the 
surface  is  proportional  to  the  cosine  of  the  angle  between  the  light  source  and  the  surface 
normal. 

Using  a  coordinate  system  centered  on  the  sphere,  let  the  light  source  be  <t>  degrees  above  the 
X-Y  plane  and  tp  degrees  from  the  X-axis.  Let  the  viewer  be  located  on  the  Z-axis.  Define  three 
constants  Cl,  C2,  C3  as 

Cl  =  cos(4>)  *  cos (xp) 

C2  =  cos (<(>)  *  sin(rp) 

C3  =  sin(V') 

The  gradient  angle  function  GA(x,y,z)  (where  x2  +  y2  z*  =  R‘)  for  a  point  on  the  sphere 
illuminated  by  the  light  source  is 

GA(x,y,z)  =  tan-1((C2— C3(y/z))/(Cl— C3(x/z))) 

Note  that  for  the  light  source  directly  overhead  (4>  =  90), 

GA(x,y,z)  =  tan_1(y/x) 

The  gradient  angle  transform  of  a  sphere  can  be  used  to  determine  the  approximate  location  (o 
and  ip)  of  the  light  source.  A  histogram  of  the  gradient  angle  over  the  sphere  has  a  maximum 
at  a  position  corresponding  to  the  angle  t£>,  while  the  difference  between  the  maximum  and 
minimum  is  inversely  related  to  d.  This  relation  has  proved  to  hold  adequately  for  actual 
spheres  and  is  used  for  parameter  extraction  in  the  sphere  detector. 


One  significant  difference  between  the  ideal  reflective  behavior  described  above  and  that  of 
actual  spheres  is  specular  reflection.  This  is  a  brighter  area  on  the  sphere  in  the  region  where 
the  surface  normal  bisects  the  angle  between  the  observer  and  the  light  source  The  image  of 
the  sphere  can  thus  be  divided  into  regions  of  shadow,  matte-reflectivity,  and  specular- 
reflectivity;  the  boundaries  between  these  areas  can  be  determined  from  a  histogram  of  the 
sample  sphere. 

Figure  1(a)  is  a  group  of  synthetically-generated  spheres.  The  direction  of  the  light  source  (b) 
varies  across  the  rows,  while  the  height  of  the  light  source  (^)  decreases  down  the  columns 
Figure  1(b)  is  the  gradient  angle  transform  of  the  image 

Figure  2  is  a  series  of  histograms  of  the  gradient  angle  transform  of  a  synthetic  sphere  as  the 
angle  <t>  varies.  Note  that  the  location  of  the  histogram  peak  (taking  averaged  values  and 
ignoring  the  spikes)  is  related  to  <j>.  Since  the  histogram  values  run  from  0  to  255,  to  get  the 
degree  value  corresponding  to  any  point  multiply  that  value  by  360/255.  We  have  found  that  d 
can  be  determined  to  within  15  degrees  using  the  histogram  peak. 

3.  SPHERE  DETECTION  AND  SEGMENTATION 

The  sphere  detector  and  segmenter  requires  as  input  the  following: 

-  the  image  to  be  examined 

-  the  gradient  angle  transform  of  the  image 

-  a  sample  feature 

-  (optional)  points  of  possible  detected  spheres 

The  first  step  in  the  processing  is  parameter-extraction.  The  program  examines  the  sample 
feature  to  determine  the  approximate  position  of  the  sun,  the  cutoffs  between  various  intensities 
on  the  sphere,  and  similar  parameters. 

The  next  step  is  feature  detection,  which  is  done  in  one  of  two  ways.  If  some  other  program 
(such  as  a  correlation  algorithm)  has  already  been  used  for  feature  detection,  the  program  uses 
its  results  as  possible  features.  Otherwise,  the  program  performs  a  simple  pattern  search  of  the 
image  to  detect  possible  features. 

The  third  step  is  the  segmentation-validation  loop.  Within  this  loop,  the  program  selects  a 
possible  feature  and  uses  angle-based  segmentation  to  create  a  candidate  region.  This  region  is 
processed  for  parameters,  which  are  compared  to  those  of  the  sample  feature.  The  region  is 
then  classified  as  POSITIVE  or  NEGATIVE  and  stored.  The  segmentation-validation  loop 
repeats  until  all  detected  features  are  classified. 


3.1.  Parameter  Extraction 

The  parameter-extraction  step  of  processing  uses  the  sample  feature  to  compute  statistical 
information  about  the  sphere.  This  information  is  used  in  both  the  feature  detection  and 
feature  validation  steps. 

The  area,  perimeter.  X  and  ’ll'  centroids,  and  radius  of  the  given  feature  are  first  extracted 
Histograms  of  the  original  image  and  the  gradient  angle  transform  over  the  feature  region  are 
then  computed  The  histogram  bins  each  contain  four  (consecutive)  grey  levels;  since  the  images 
we  use  have  256  grey  levels  (8-bits),  each  histogram  has  64  bins  Each  histogram  entry  is 


replaced  l.\  the  airra<r  uf  the  entr\  aini  1 1  >  neighbors  l  h  i  r~  smooths  the  hologram  and  aids  m 
the  parameter  extraction 

The  maximum  of  the  gradient  angle  histogram  is  located  and  used  to  determine  the 
approximate  direction  of  the  sun,  as  described  in  section  -  3  Tins  value  is  then  displayed  arid 
the  user  is  allowed  to  change  it  if  desired  The  original  image  histogram  is  examined  to 
determine  approximate  thresholds  between  the  shadowed,  matte-reflecting,  and  specular- 
reflecting  areas  of  the  sphere  The  shadowed  area  is  taken  to  run  from  0  to  the  first  local 
minimum  as  the  histogram  index  increases,  while  the  specular-reflecting  area  is  taken  to  run 
from  255  to  the  first  local  minimum  as  the  histogram  index  decreases  After  these  are  computed 
the  user  is  allowed  to  change  the  values  if  desired 

Parameters  for  a  "pattern-search"  of  the  image  are  then  computed.  Consider  a  sphere 
thresholded  to  three  levels  corresponding  to  the  shadowed,  matte-reflecting,  and  specular- 
reflecting  regions.  Moving  along  a  line  through  the  center  of  the  sphere  in  the  direction  away 
from  the  light  source,  these  three  regions  will  be  encountered  in  the  order  "specular,  matte, 
shadow"  (i.e.,  a  consecutive  series  of  points  on  the  line  consisting  of  one  or  more  specular  points, 
one  or  more  matte  points,  and  one  or  more  shadow  points).  We  use  this  both  to  extract  the 
parameters  and  to  detect  spheres. 

Lines  parallel  to  the  direction  of  the  sun  are  drawn  across  the  sample  feature.  Each  line  is 
examined  for  the  pattern  "specular,  matte,  shadow".  The  widths  of  the  three  regions  are  stored. 
After  the  whole  of  the  sample  feature  has  been  so  processed,  the  stored  data  is  evaluated  to 
obtain  minimum  and  maximum  lengths  for  each  of  the  three  regions  as  well  as  bounds  on  the 
ratios  of  the  lengths. 

3.2.  Detection 

If  another  program  has  been  used  to  locate  possible  spheres,  these  results  are  used.  We  have 
had  great  success  with  a  correlation  (using  sum  of  squared  differences)  of  the  sample  feature  in 
the  gradient  angle  space  against  the  whole  image.  This  has  proved  very  effective  in  detecting 
spheres  in  that  it  both  detects  all  spheres  of  approximately  the  same  size  as  the  feature  and 
makes  very  few  false  detections. 

If  no  such  program  has  been  used,  a  pattern-search  is  performed  to  detect  spheres.  The 
parameters  previously  computed  are  used. 

Scans  of  the  image  at  the  sun  angle  plus  15  degrees  and  the  sun  angle  minus  15  degrees  are 
made.  When  the  pattern  "specular  matte  shadow"  is  encountered  with  the  lengths  and  length 
ratios  of  the  three  regions  falling  within  the  parameters  previously  determined,  the  points 
making  up  the  pattern  are  marked  POSSIBLE.  Points  marked  by  both  scans  which  are  also  in 
the  specular  range  are  taken  as  candidate  points. 


3.3.  Segmentation 


In  the  segmentation  step,  the  brightest  remaining  candidate  point  is  selected  as  the  initial  point 
A  region-grower  based  on  image  intensity  is  first  used  Lsing  standard  methods  of  adding  point5 
to  the  region  if  thev  meet  the  selection  criteria,  this  grower  produces  a  region  around  the 
candidate  point  for  winch  the  image  intensity  falls  within  the  specular  range 
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The  second  region  grower,  based  on  gradient  angle  continuity,  is  then  used  to  expand  the  region 
Points  on  the  perimeter  of  the  region  which  border  at  least  three  points  already  in  the  region 
are  considered.  If  the  minimum  and  maximum  gradient  angle  of  the  bordering  points  differ  by 
less  than  60  degrees  and  the  gradient  angle  of  the  point  under  consideration  lies  in  the  range 
minangle-15  ..  maxangle+15{,  the  point  is  added  to  the  region.  Otherwise  the  point  is  marked 
as  unusable.  This  process  is  continued  until  no  more  points  can  be  added  to  the  region.  The 
values  60  degrees  and  15  degrees  used  above  were  established  through  experimentation. 


3.4.  Validation 

The  region  created  by  the  segmentation  step  is  processed  for  the  same  statistics  as  in  the 
parameter-extraction  step.  The  region  is  then  "graded"  by  applying  a  number  of  tests  designed 
to  evaluate  its  sphericity.  If  the  region  fails  too  many  of  these  tests,  it  receives  a  NEGATIVE 
evaluation;  otherwise  it  receives  a  POSITIVE  evaluation. 

The  region  is  then  displayed  to  allow  the  user  to  change  the  evaluation  if  desired.  Regions  with 
a  final  evaluation  of  POSITIVE  are  saved,  and  any  candidate  points  within  or  bordering  the 
region  are  removed  from  further  consideration.  The  latter  step  is  to  avoid  duplication  of 
regions. 

The  validation  tests  include  the  following: 

--  check  that  the  ratio  of  the  area  (points  within  region) 
and  perimeter  (points  bordering  region)  are  consistent 
with  that  of  a  sphere 

--  check  that  the  angle  of  the  sun  computed  from  the  region 
agrees  with  the  actual  angle 

--  check  that  the  shadow,  matte,  and  specular  thresholds 
agree  with  those  of  the  original  feature 

—  check  that  the  percentage  areas  of  shadow,  matte,  and 
specular  zones  agree  with  those  of  the  original  feature 

Segmentation  and  validation  take  place  together  within  a  loop;  points  are  selected,  regions 
segmented,  and  validation  checks  made  until  no  further  candidate  points  remain. 


4.  RESULTS 

Figures  3  through  6  show  the  results  of  a  program  run.  Figure  3  shows  the  initial  image,  from 
which  spheres  are  to  be  extracted.  This  image  was  first  processed  to  get  the  gradient  angle 
transform,  and  a  sample  feature  (the  sphere  toward  the  upper  left)  was  extracted 

Figure  4  shows  the  results  of  a  correlation  detector.  The  sample  feature's  gradient  angle  was 
correlated  over  the  whole  image,  using  a  squared-differences  computation  Points  of  low- 
intensity  correspond  to  good  correlation:  in  the  figure,  these  appear  in  red.  We  were  able  to  use 
a  simple  correlator  on  the  spheres  because  they  remain  unchanged  if  rotated  More 
sophisticated  methods  are  required  for  non-symmetric  objects 

Figure  shows  the  results  of  detection  by  pattern  match  as  produced  by  the  procram  I  icur*  ..  ,i 
shows  t  fie  actual  pattern  produced  as  described  earlier,  while  figure  afb)  shows  the  delected 
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points.  The  program  used  these  detected  points  in  its  search. 

Figure  6  depicts  the  areas  which  the  program  classified  as  spheres,  highlighted  in  red.  Note  that 
all  spheres  have  been  detected,  and  that  no  false  positive  classifications  have  been  made. 


5.  CONCLUSIONS 

The  motivation  for  this  work  rests  with  the  need  for  developing  algorithms  which  detect  and 
segment  specific  types  of  objects  in  aerial  photographs.  When  both  the  number  and  size  of  the 
photographs  are  large  this  task  becomes  increasingly  difficult  to  be  performed  by  humans,  who 
get  tired  and  lose  concentration. 

Our  algorithm  is  specialized  for  seeking  out  spherical  objects  having  smooth  reflecting  surfaces. 
It  requires  a  sample  object  whose  analysis  is  used  to  compute  var  parameters  involved  in  the 
detection  and  validation  process.  All  parameters  are  subject  to  human  review  and  correction,  if 
necessary. 

The  principal  means  of  detection  is  the  continuous  nature  of  the  gradient  angle.  There  are 
strong  indications  that  a  similar  approach  may  be  useful  for  many  other  important  aerial 
features.  Although  the  continuity  of  the  gradient  angle  seems  to  be  an  important,  physically 
relevant  characteristic,  it  has  received  only  limited  attention  in  the  past. 


ft 
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Figure  2(c).  <(>  —  144  degrees,  or  2/5 
of  the  total  circle. 


Figure  2(d).  <t>  -  216  degrees,  or  3/5 
of  the  total  circle. 
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Figure  5(b).  Points  detected  by 
pattern  search  (marked  in  both  searc 
directions  and  high-intensity) 


Figure  6.  Detected  spherical 


